Computing devices continue to become more ubiquitous to daily life. They take the form of computer desktops, laptop computers, tablet computers, hybrid computers (2-in-1s), e-book readers, mobile phones, smartphones, wearable computers (including smartwatches, smart glasses/headsets), global positioning system (GPS) units, enterprise digital assistants (EDAs), personal digital assistants (PDAs), game consoles, and the like. Further, computing devices are being incorporated into vehicles and equipment, such as cars, trucks, farm equipment, manufacturing equipment, building environment control (e.g., lighting, HVAC), and home and commercial appliances.
Computing devices generally consist of at least one processing element, such as a central processing unit (CPU), some form of memory, and input and output devices. The variety of computing devices and their subsequent uses necessitate a variety of interfaces and input devices. One such input device is a touch sensitive surface such as a touch screen or touch pad wherein user input is received through contact between the user's finger or an instrument such as a pen or stylus and the touch sensitive surface. Another input device is an input surface that senses gestures made by a user above the input surface. A further input device is a position detection system which detects the relative position of either touch or non-touch interactions with a non-touch physical or virtual surface. Any of these methods of input can be used generally for drawing or inputting text. When user input is text the user's handwriting is interpreted using a handwriting recognition system or method.
One application of handwriting recognition in non-portable and portable computing devices, such as smartphones, phablets and tablets, is for the input of text into various applications run by the computing devices in a manner similar to that traditionally done with keyboard, either physical or virtual. However, unlike keyboard input which is governed by strict layout rules, such as text entry on visible or invisible lines in accordance with the position of a displayed cursor, handwriting can be input virtually anywhere on the input surface of the device. While this substantially unconstrained input has some effect on the recognition accuracy of the handwriting, currently available advanced recognition algorithms are generally able to deal with such relatively ‘free’ positioning of textual characters.
Of more effect is a user's ability to interact with the input once it has been rendered as so-called ‘digital ink’ on the display. This is because, unlike typed or ‘typeset ink’ text which has characters relatively located in a uniform and known manner, the characters of the digital ink text are not uniform or in relatively known positions, generally due to the irregularity inherent in handwriting. Accordingly, conventional systems and methods typically only allow further interaction to be made on the typeset ink rendered from the recognized text, which is generally done at a different position of the display from the area where input is allowed.
It is desired by users however to be able to interact directly with the handwriting as well, such as to edit the input (e.g., add or delete characters or words), so that the creative flow of handwriting input is maintained. Some available applications provide alignment and scaling of the digital ink through normalization and the like. However, these do not take into account certain geometrical features of the handwriting and therefore the scaling and aligning is not properly performed to allow full interaction or faithful reproduction of the original digital ink. Further, they do not use recognition to provide these operations, and as such do not provide interaction capabilities within the input signal. Accordingly, editing operations can only be performed at the word level, and not at the character level, for example.